{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Matching"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"import math\n",
"import os\n",
"import sys\n",
"\n",
"sys.path.insert(0, os.path.abspath('/data/autocnet'))\n",
"\n",
"import autocnet\n",
"from autocnet import CandidateGraph\n",
"\n",
"# The GPU based extraction library that contains SIFT extraction and matching\n",
"import cudasift as cs\n",
"\n",
"# A method to resize the images on the fly.\n",
"from scipy.misc import imresize\n",
"\n",
"%pylab inline\n",
"figsize(16,4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## CandidateGraph -> Custom Extraction Func -> Matching"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"a = 'AS15-P-0111_CENTER_LRG_CROPPED.png'\n",
"b = 'AS15-P-0112_CENTER_LRG_CROPPED.png'\n",
"\n",
"adj = {a:[b],\n",
" b:[a]}\n",
"\n",
"cg = CandidateGraph.from_adjacency(adj)\n",
"\n",
"# Enable the GPU\n",
"autocnet.cuda(enable=True, gpu=0)\n",
"\n",
"# Write a custom keypoint extraction function - this could get monkey patched onto the graph object...\n",
"def extract(arr, downsample_amount=None, **kwargs):\n",
" total_size = arr.shape[0] * arr.shape[1]\n",
" if not downsample_amount:\n",
" downsample_amount = math.ceil(total_size / 12500**2)\n",
" shape = (int(arr.shape[0] / downsample_amount), int(arr.shape[1] / downsample_amount))\n",
" # Downsample\n",
" arr = imresize(arr, shape, interp='lanczos')\n",
" \n",
" npts = max(arr0.shape) / 3.5\n",
" sd = cs.PySiftData(npts)\n",
" cs.ExtractKeypoints(arr, sd, **kwargs)\n",
" kp, des = sd.to_data_frame()\n",
" kp = kp[['x', 'y', 'scale', 'sharpness', 'edgeness', 'orientation', 'score', 'ambiguity']]\n",
" kp['score'] = 0.0\n",
" kp['ambiguity'] = 0.0\n",
" \n",
" return kp, des, sd, downsample_amount, arr\n",
"\n",
"arr0 = cg.node[0].geodata.read_array()\n",
"kp0, des0, sd0, downsample_amount0, arr0 = extract(arr0, thresh=1)\n",
"\n",
"arr1 = cg.node[1].geodata.read_array()\n",
"kp1, des1, sd1, downsample_amount1, arr1 = extract(arr1, thresh=1)\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(5, 5)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# How much downsampling?\n",
"downsample_amount0, downsample_amount1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that the keypoints have been extracted it is time to apply the GPU based matcher. Since this is working with the lower level API, the correspondences are read from the GPU and then piped back. This is technical inefficient, but fast enough that saving a few lines of code managing GPU memory is worth it."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sd0 = cs.PySiftData.from_data_frame(kp0, des0)\n",
"sd1 = cs.PySiftData.from_data_frame(kp1, des1)\n",
"# Apply the matcher\n",
"cs.PyMatchSiftData(sd0, sd1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The matches DataFrame\n",
"\n",
"Matches are stored as [Pandas DataFrames](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) meaning that all functionality available to a Pandas DataFrame is available to the matches. This includes:\n",
"\n",
"- Visualization\n",
"- Sorting\n",
"- SQL style queries\n",
"- Split-Apply-Combine Operations\n",
"- Statistical Analysis"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" x | \n",
" y | \n",
" scale | \n",
" sharpness | \n",
" edgeness | \n",
" orientation | \n",
" score | \n",
" ambiguity | \n",
" match | \n",
" match_xpos | \n",
" match_ypos | \n",
" match_error | \n",
" subsampling | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 9528.347656 | \n",
" 195.259979 | \n",
" 20.769804 | \n",
" 3.171909 | \n",
" 4.874205 | \n",
" 191.425842 | \n",
" 0.952312 | \n",
" 0.998823 | \n",
" 1969 | \n",
" 7409.627930 | \n",
" 464.366821 | \n",
" 0.0 | \n",
" 0.0 | \n",
"
\n",
" \n",
" | 1 | \n",
" 4080.437012 | \n",
" 66.662041 | \n",
" 16.442673 | \n",
" -1.066230 | \n",
" 5.038958 | \n",
" 272.796051 | \n",
" 0.978477 | \n",
" 0.999569 | \n",
" 351 | \n",
" 7551.666992 | \n",
" 42.322033 | \n",
" 0.0 | \n",
" 0.0 | \n",
"
\n",
" \n",
" | 2 | \n",
" 5775.711914 | \n",
" 140.684418 | \n",
" 21.859934 | \n",
" 2.915606 | \n",
" 4.006656 | \n",
" 247.030746 | \n",
" 0.936110 | \n",
" 0.949781 | \n",
" 21 | \n",
" 3847.362793 | \n",
" 140.941910 | \n",
" 0.0 | \n",
" 0.0 | \n",
"
\n",
" \n",
"
\n",
"